Merging content

ABSTRACT

A device may detect an event, determine a display layout in accordance with the event, receive input video streams, merge two or more of the input video streams into a high definition output stream in accordance with the display layout, and send the high definition output stream to a content presentation device.

BACKGROUND

Many of today's entertainment or communication-related electronicdevices rely on receiving, transmitting, and/or using streaming digitaldata or content. For example, a set-top box may receive broadcasttelevision programs and/or video-on-demand (VOD) that is streamed from acontent provider. A personal computer may receive a stream of a videoclip over the Internet. A soft phone may receive streaming audio dataover a real-time transport protocol (RTP) link/channel that isestablished over an Internet Protocol (IP) network.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system in which an exemplary contentmerging device may be implemented;

FIGS. 2A and 2B illustrate exemplary operation of the content mergingdevice of FIG. 1;

FIG. 2C illustrates another exemplary operation of the content mergingdevice of FIG. 1;

FIG. 3 shows an exemplary network in which concepts described herein maybe implemented;

FIG. 4A is a block diagram of exemplary hardware components of thecontent merging device of FIG. 3;

FIG. 4B is a block diagram illustrating flow of content streams throughthe components of FIG. 4A;

FIG. 4C is a block diagram illustrating some of the exemplary functionsof the components of FIG. 4B;

FIG. 5 is a block diagram of exemplary functional components of thecontent merging device of FIG. 3;

FIG. 6 is a flow diagram of an exemplary process associated withchanging a layout for displaying content streams; and

FIGS. 7A and 7B illustrate an example associated with merging content.

DETAILED DESCRIPTION

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements. As used herein, the terms “playing content” mayrefer to decoding encoded content, and outputting the decoded content toa presentation device (e.g., television). The encoded content may beobtained from a local media (e.g., hard disk drive) or received from aremote device over a network.

As described herein, a device may receive streams of content (e.g.,video streams, audio streams, etc.) and merge the streams to be outputon one or more devices (e.g., display screens, televisions, speakers,etc.). FIG. 1 illustrates an exemplary system in which a content mergingdevice 102 may be implemented. As shown, system 100 may include contentmerging device 102, a set-top box 104, a digital video recorder (DVR)106, a digital television 110, and network 112 (e.g., an InternetProtocol (IP network)).

In FIG. 1, content merging device 102 may receive different video inputsfrom one or more of set-top box 104, DVR 106, and/or network 112.Furthermore, content merging device 102 may combine/merge the receivedvideo streams into a single stream, and send the combined stream todigital television 110. Content merging device 102 may combine thereceived video streams such that when the combined stream is displayedat digital television 110, each of the received videos is displayed inan area that is set in accordance with a default or user-specifiedconfiguration of content merging device 102.

FIGS. 2A and 2B illustrate exemplary operation of content merging device102. Assume that Jacque and Sunny have installed, in their respectivehomes 200 and 220, a content merging system 100. Jacque is watching adocumentary film Tigers 202 that is playing on his DVR 106 and beingoutput to his digital television 110. Sunny is watching a cabletelevision movie The Three Musketeers 222 that is playing at her set-topbox 104 and being output to her digital television 110.

While watching the film Tigers 202, Jacque decides to place a videophonecall to Sunny via a soft videophone installed on content merging device102. Using a remote control, Jacque dials Sunny's videophone number. Thesoft videophone shows a status message 204 of the call on Jacque'stelevision 110. When Jacque places the call, a soft videophone oncontent merging device 102 at Sunny's home 220 displays a text message224 on Sunny's digital television 110, indicating an incoming call.

When Sunny receives Jacque's call, the soft videophones at Sunny andJacque's content merging devices 102 establish communication sessionsover network 112. The soft videophone at Jacque's content merging device102 receives a video of Jacque via a camera 206 included in Jacque'sdigital television 110 and streams the video to Sunny's content mergingdevice 102.

In response, Sunny's content merging device 102 merges the receivedvideo of Jacque 228 and the video of Sunny 230 from camera 226 into asingle video stream. The right side of FIG. 2B shows Sunny's digitaltelevision 110 displaying the combined video stream, with Sunny's video230 merged within Jacque's video 228. Similarly, Jacque's contentmerging device 102 merges the received video of Sunny 208 and the videoof Jacque 210 from camera 206 into a single video stream. The left sideof FIG. 2B shows Jacque's digital television displaying the combinedvideo stream, with Jacque's video 210 merged within Sunny's video 208.

FIG. 2C illustrates another exemplary operation of content mergingdevice 102. In this example, assume that Jacque wants to send the videostream of Tigers 202 to Sunny. Jacque may activate a menu and select,for example, via a remote control, to send a combined video stream ofTigers 202 and video 228 to Sunny. The right side of FIG. 2C depictsSunny's digital television 110 showing the combined video stream (fromJacque's content merging device 102) that includes the video stream 228for Tigers and the video stream 228 of Jacque.

In the above illustrations associated with FIGS. 2A through 2C, contentmerging device 102 merges video streams in the context of videotelephony, videoconferencing, and/or video telepresence. In someimplementations, content merging device 102 may include a user interface(e.g., a graphical user interface (GUI) that allows a user to select anumber of streams to be combined (e.g., streams from set-top box 104,DVR 106, a multitude of video phone callers in network 112, an audiodevice, a computer, etc.). The merged or combined streams may be outputto different output devices or components (e.g., a hard disk drive(e.g., to store the combined stream), a network interface (to send thecombined stream over a network), etc.).

In the above, content merging device 102 may be interconnected todifferent media devices that would otherwise be connected to television110. Thus, using content merging device 102 in system 100 may reduce thenumber of cables that are connected to digital television 110. Inaddition, because content merging device 102 may be a separate devicefrom set-top box 104, content merging device 102 may be manufactured andsold independently from set-top box 104 vendors. Further, contentmerging device 102 may allow video telepresence, video teleconference,and/or videophone services to be provided independently from contentproviders. For example, a videophone service may be provided by a vendorindependently from a service provider for a cable network.

FIG. 3 shows an exemplary network 300 in which concepts described hereinmay be implemented. As shown, network 300 may include a content mergingdevice 102, content delivery device 304, content player device 306,mobile device 308, content presentation device 310, content providerdevice 312, content storage device 314, and network 316.

Content merging device 102 may execute one or more applications thatreceive content streams from devices over network 316, content deliverydevice 304, content player device 306, mobile device 308 (e.g., ahandset), and/or another device. In addition, content merging device 102may merge or combine the received streams into one or more outputstreams and send the output streams to mobile device 308, devices innetwork 316, and/or content presentation device 310.

Content delivery device 304 may receive encoded content over network316, decode the content, and/or output the decoded content to contentmerging device 302 and/or content presentation device 310. In someimplementations, content delivery device 304 may obtain the content froma content provider device 312, which may be part of a contentdistribution system (e.g., a video/audio broadcast system,content-on-demand distribution system, and/or local storage component(e.g., a hard disk drive)).

In addition to outputting decoded content to content merging device 102and/or content presentation device 310, content delivery device 304 mayperform actions in response to commands (e.g., fast forward, pause,select a program/content, etc.) that are issued from a remote control orfrom content merging device 102.

Examples of content delivery device 304 may include a set-top box or acomponent (e.g., a cable card) that plugs-into a host device (e.g., adigital video recorder, a personal computer, a television, a stereosystem, etc.) and allows the host device to display multimedia content(e.g., contents on digital cable television channels). Although contentdelivery device 304 can be implemented as different types of devices(e.g., a set-top-box, computer, cable card, etc.), in the following,content delivery device 304 is described in terms of a set-top box.

Content player device 306 may receive encoded or decoded content from alocal, external storage (e.g., a digital video disc (DVD)), decode thereceived content if the content is encoded, and/or output the decodedcontent to content merging device 306 and/or content presentation device310. In one implementation, content player device 306 may output thestream, via a high definition multimedia interface (HDMI) cable, tocontent merging device 102.

Mobile device 308 may include a handset, cellular phone, and/or anothertype of communication device. In one implementation, mobile device 308may function as a cellular telephone. In some instances, mobile device308 may place an outgoing call on behalf of a user over a wireless link(e.g., Bluetooth) via content merging device 102. In anotherimplementation, mobile device 308 may operate as a wireless/cordlesshandset for content merging device 102. In such an implementation,content merging device 102 may operate as a base station for mobiledevice 308.

Content presentation device 310 may include, for example, a television,display, speakers, etc. Content presentation device 310 may receivedigital and/or analog content from content merging device 102 andpresent the content via its display screen and/or its speakers.

Content provider device 312 may obtain content (e.g., a video program,audio program, advertisement, web page, etc.) from content storagedevice 314 and/or send the content to content delivery device 304,content merging device 102, and/or content presentation device 310 overnetwork 316. Content storage device 314 may retrieve content (e.g., amovie, a music file, an advertisement, text, web page, etc.) and sendthe content to content provider device 312 when content storage device314 receives a request for content from content provider device 312.

Network 316 may include one or more wired and/or wireless networks thatare capable of exchanging information, such as voice, video, documents,multimedia, text, etc. For example, network 316 may include one or morepublic switched telephone networks (PSTNs) or another type of switchednetwork. Network 316 may also include a number of transmission towersfor receiving wireless signals and forwarding the signals toward theintended destination. Network 316 may further include one or more packetswitched networks, such as an Internet protocol (IP) based network, alocal area network (LAN), a wide area network (WAN), a personal areanetwork (PAN), an intranet, the Internet, or another type of networkthat is capable of exchanging information. In FIG. 3, devices 302through 314 may communicate directly with one another or communicatewith one another via network 316

Depending on the implementation, network 300 may include additional,fewer, different, or different arrangement of devices than thoseillustrated in FIG. 3. For example, in one implementation, network 300may include additional content merging devices, content deliverydevices, content presentation devices, etc. In another example, network300 may include a personal computer or a laptop computer that iscommunicating with content merging device 102 via local networkconnections. In FIG. 3, network 300 does not show some elements ofnetwork, such as routers, bridges, switches, gateways, servers, etc.,that route data in network 300.

FIG. 4A is a block diagram of exemplary hardware components 402-420 ofcontent merging device 102. As shown, content merging device 102 mayinclude a processor 402, codec engine 404, memory 406,picture-in-picture (PIP) module 408, memory 410, video receiver 412,video transmitter 414, signal processor 416, base station 418, andcommunication interface 420.

Processor 402 may include one or more processors, microprocessors,Application Specific Integrated Circuits (ASICs), Field ProgrammableGate Array (FPGA) devices, and/or other processing components capable ofprocessing data and/or controlling content merging device 102. In someimplementations, processor 402 may include one or more embeddedprocessors for high-speed video processing and/or communication. Asshown, processor 402 may coordinate data traffic between communicationinterface 420, external device (e.g., content player device 306 viaUniversal Serial Bus (USB)), codec engine 404, PIP module 408, etc. Inone implementation, processor 402 may be coupled to a high speed, lowpower memory, such as a double data rate 2 memory, a synchronous dynamicrandom access memory (DDR2 SDRAM) or a storage device, such as NANDflash storage.

Codec engine 404 may include one or more codec components (e.g.,processors for encoding/decoding data). Each codec component mayimplement a specific encoding/decoding scheme. In addition, codec engine404 may manage resources for the codec components (e.g., allocatememory). Memory 406 may provide a dynamic memory pool from which codecengine 404 may allocate memory for a codec component. Memory 406 mayinclude, for example, mobile double data rate (MDDR) synchronous dynamicrandom access memory (SDRAM) or DDR SDRAM. A MDDR may consume less powerthan a number of other types of SDRAM, and provide for greater powerefficiency.

PIP module 408 may include one or more processors for merging the framesof received content streams (e.g., each of the frames of streams fromcontent delivery device 304, content player device 306, network 316,etc.) into frames of an output stream in accordance with a layout. Forexample, assume that PIP module 408 receives two content streams (e.g.,a content stream from a videophone call and another stream from a camera(not shown)) via processor 402 and codec engine 404, respectively. Inresponse, PIP module 408 may place frames or portions of frames of thereceived streams within each frame of an output stream. The specificlayout (e.g., how the received frames are combined to produce an outputframe) may be dependent on a specific application/logic that is runningon content merging device 102. PIP module 408 may generate and writeeach of the frames of the output stream to memory 410.

Memory 410 (e.g., SDRAM) may temporarily store each frame of the outputstream until the frame is transmitted via video transmitter 414. Videoreceiver 412 may receive video and/or audio streams (e.g., HDMI stream)that are output from content delivery device 304, decode the streams(e.g., the stream may be high content digital content protection (HDCP)encoded) and relay the decoded video stream to PIP module 408, where thevideo stream is merged with other streams. Video transmitter 414 mayencode (e.g., HDCP encode) and stream frames that are buffered in memory410, along with an audio stream, as an output stream (e.g., HDCP encodedHDMI stream) to content presentation device 310.

Signal processor 416 may receive an audio signal from processor 402,process the audio signal (e.g., perform audio echo cancellation (AEC))and forward the audio signal to base station 418. Base station 418 mayrelay the audio signal over a wireless communication link to mobiledevice 308. In this implementation, mobile device 308 may operate as ahandset of a home telephone system.

Communication interface 420 may enable content merging device 102 tocommunicate with other devices and/or systems in network 316 or network300. For example, network interface 420 may communicate via a network,such as the Internet, a terrestrial wireless network (e.g., a WLAN,WiFi, etc.), a public switched telephone network (PSTN), asatellite-based network, etc. Communication interface 420 may include amodem, an Ethernet interface (e.g., 10/100/1000 Mb) to a LAN, and/or aninterface for connecting network device 420 to other devices (e.g., viaa Bluetooth interface).

Depending on the implementation, content merging device 102 may includeadditional, fewer, or different components than the ones illustrated inFIG. 4A (e.g., plug-in cards for additional serial buses, hard diskdrives, display screens (e.g., liquid crystal display (LCD), camera,microphone, speakers, infrared signal detector (e.g., for a remotecontrol), microphone, etc.). In addition, for simplicity, FIG. 4A doesnot show other components of content merging device 102, such asadditional memories (e.g., read-only-memory (ROM)), local cache, etc.).

FIG. 4B is a block diagram illustrating flow of content streams throughsome of the components of content merging device 102. As shown, contentstreams may arrive from content delivery device 304, network 316, andcontent player device 306. Although content merging deice 302 mayreceive other streams (e.g., a stream from a camera installed on contentpresentation device 310), FIG. 4B does not show them for simplicity.

In FIG. 4B, the content streams from network 316 and content playerdevice 306 are routed by processor 402 to codec engine 404, decoded atcodec engine 404, and sent to PIP module 408. PIP module 408 combinesthe streams from processor 402 and content delivery device 304 (viavideo receiver 412), and outputs the combined stream to contentpresentation device 310 (via video transmitter 414).

In FIG. 4B, some data paths are not shown for simplicity. For example,processor 402 may route the output stream from PIP module 408 to codecengine 404, to obtain an encoded stream. Processor 402 may send theencoded stream to network 316 via communication interface 420. Inanother example, processor 402 may receive audio input via a microphoneand route the audio signal to signal processor 416. Each data path maydepend on the applications, logic, and/or software components that areinstalled and/or running on content merging device 102.

FIG. 4C is a block diagram highlighting some of the exemplary high speedfunctions of the components of FIG. 4B. As discussed above, videoprocessing functionalities of content merging device 102 may bedistributed over hardware components, some of which are shown in FIG. 4Cas processor 402, codec engine 404, PIP module 408, video receiver 412,and video transmitter 414. Some of these functions may be performed athigh speeds.

For example, processor 402 may direct and/or regulate high speed trafficbetween network 216, codec engine 404, PIP module 408, and/or externaldevices. Codec engine 404 may perform high-speed encoding or decoding422 of data from network and/or other external devices. Codec engine 404may receive/output the data via processor 402 and/or PIP module 408.

PIP module 408 may perform high speed scaling, cropping, and/ortranslating 424 of graphical data in accordance with a predeterminedlayout, for merging and/or combining video data. Video receiver 412 andtransmitter 414 may perform high speed decoding 426 and encoding 428 ondata received from an external device and PIP module 408 (e.g., HDCPdecoding/encoding), respectively.

By performing the encoding/decoding functions via codec engine 404, PIPmodule 408, video receiver 412, and/or video transmitter 414 asillustrated in FIG. 4C, content merging device 404 may avoid bottlenecksin high definition video processing (e.g., merging HDMI video with othervideo streams) and combine/manipulate the high definition images at highspeeds without suffering from, for example, freezing, stopping, etc. Inimplementations that include a sufficiently fast high-speed generalprocessor(s) and other components to support such processors (e.g., databus), some of the components (e.g., codec engine 404, PIP module 408,etc.) in FIG. 4A may be replaced with software modules/components (e.g.,a software codec engine).

FIG. 5 is a block diagram of exemplary functional components of contentmerging device 102. As shown, content merging device 102 may includevideo layout logic 502, video conferencing (or video telepresence) logic504, DVR logic 506, phone logic 508, content delivery logic 510, weblogic 512, thread management logic 514, audio engine 516, and videodriver 518. All or some of these logic components may be implemented byprocessor 402, PIP module 408, and/or codec engine 404 executingsoftware instructions stored in a memory (e.g., a memory coupled toprocessor 402).

Depending on the implementation, content merging device 102 may includeadditional, fewer, different, or different arrangement of componentsthan those illustrated in FIG. 5. For example, in one implementation,one component may include functionalities of one or more components(e.g., video layout logic 502 is included in video conferencing logic504). In addition, for simplicity, FIG. 5 does not show a number ofcomponents, such as a graphical user interface (GUI) components, anoperating system (e.g., Linux), a hardware abstraction layer, networkcommunication software, applications/scripts (e.g., http server, XMLparser, common gateway interface (CGI), dynamic host configurationprotocol (DHCP) client, WiFi client, etc.), etc.

Video layout logic 502 may specify a layout of windows or portions of adisplay for displaying different video streams received at contentmerging device 102. In one implementation, video layout logic 502 mayreceive user input, and based on the user input, define the layout. Forexample, video layout logic 502 may receive user input (via a GUI) whichspecifies that a video stream from content player device 306 (e.g., DVR106) is to be shown in a small area within another area that shows avideo stream from content delivery device 304.

In addition, based on different content streams that are received andbased on different states of logic 504-512, video layout logic 502 mayset a current layout for PIP module 408. PIP module 408 may mergedifferent video streams in accordance with the current layout, such thateach of the merged streams is displayed in a corresponding area in thecurrent layout.

Video conferencing logic 504, DVR logic 506, phone logic 508, andcontent delivery logic 510 may handle content streams from/to a softvideophone installed on content merging device 102 (not shown in FIG.5), DVR 306, mobile device 308, content delivery device 304, and/ornetwork 316. For example, assume that Jacque receives a videophone callfrom Sunny. Upon receiving the call, video conferencing logic 504 maynotify Jacque of the incoming call and display a GUI (e.g. a menu thatis overlaid on an output stream). Via the GUI, Jacque may accept thecall and notify video layout logic 502 that he is accepting the call. Ina different implementation, video layout logic 502 may actively pollvideo conferencing logic 504 (or any other logic 506-512) to identifynew streams or terminations of existing streams. Based on the result ofpolling, video layout component 502 may identify a current layout forPIP module 408.

In some implementations, each of video conferencing logic 504, DVR logic506, phone logic 508, content delivery logic 510, and web logic 512 mayreceive user/viewer input or commands, e.g., via a remote control forcontrolling a video conferencing device (e.g., a device installedexternal to content merging device 102), content player device 306,mobile device 308, content delivery device 304, and/or a browsing device(not shown) and relay the input or commands to the video conferencingdevice, DVR 306, mobile device 308, content delivery device 304, and thebrowsing device via wired/wireless communication links. In theseimplementations, logic 504-512 may indicate any new streams orterminations of content streams to video layout component 502 when aparticular input/command is transmitted to the controlled device.

Thread management logic 514 may allocate processing cycles to threadsfor handling different processes. For example, in one implementation,thread management logic 514 may allocate processing time to threads forreceiving user input to change a layout, to threads for encoding ordecoding video streams from different devices, to threads for handlingincoming videophone calls, etc.

Audio engine 516 may process audio signals. The audio signals may bereceived via a microphone that is connected to content merging device102, mobile device 308, video receiver 412, content player device 306(e.g., DVD player or compact disc player), content delivery device 304,etc. For example, in one implementation, audio engine 516 may include aband pass digital filter. Video driver 518 may include logic forcontrolling video components, such as video receiver 412, videotransmitter 414, etc.

FIG. 6 is a flow diagram of an exemplary process 600 associated withchanging a layout for displaying content streams. As indicated above,PIP module 408 may combine or merge received streams in accordance witha layout. The layout may change, depending on the state of contentmerging device 102. In an exemplary implementation, the state may changedue to different factors, such as viewer/user input, occurrence of anevent (e.g., arrival of a videophone call, termination of an ongoingcall, starting a new application that outputs different streams todifferent areas of the display screen of content presentation device310, etc.).

Process 600 that is associated with changing a layout may start with adetection of an event (block 602). The event may be driven, for example,by an application, by an external device (e.g., content delivery device304, by a user who modifies the layout via a GUI, etc. Assume, forexample, that the event is an incoming videophone call.

Content merging device 102 may determine, upon detection of an event,whether an event of a particular type has occurred (block 604). Forexample, content merging device 102 may determine whether auser-initiated event that changes a layout has occurred, a softvideophone has received an invite message to establish a communicationsession, a new video stream has been created, an ongoing videophone callterminated, etc. If the event of the particular type has not occurred(block 604—NO), process 600 may return to block 602.

If the event has occurred (block 604—YES), content merging device 102may receive input from a user (block 606). Upon the occurrence of theevent, content merging device 102 may alert or prompt the viewer foradditional input. For example, after receiving an invite message from aremote videophone caller, content merging device 102 may display, atcontent presentation device 310, a text message. The text message mayshow a caller ID and may request the user to indicate whether the userwill accept the call. In another example, content delivery device 304may begin to stream a particular television program. Upon detecting thestart of the transmission, content delivery device 304 may request thatthe user confirm whether the user will view the content.

In some implementations, performing block 606 may entail severaliterations of receiving user input and providing the user with differentoptions. For example, assume that the user has entered a menu. Via a GUIinterface, the user may select different layout options (e.g., alocation or size of a viewing window for one type of content stream,etc.). This may require the user to provide several inputs in thehierarchy of menus.

Content merging device 102 may determine whether the input indicates animmediate change in the viewing layout (block 608). For example, assumethat at block 606, the user has indicated that the user will accept anincoming videophone call. In such a case, content merging device 102 maydetermine that the viewing layout is to be changed (e.g., display thevideo stream from a remote caller and a video stream from a camera thatis connected to content merging device 102).

In another example, assume that at block 606, during the videophonecall, the user has activated a menu screen (e.g., via a remote control)and selects a streaming video from content delivery device 304 to besent to the caller. The selection may indicate to content merging device102 to modify the current viewing layout.

In yet another example, assume that the user has received, during thevideophone call, another videophone call and the user has indicated thatthe user will accept the second call. The input may indicate to contentmerging device 102 that the video stream from the second caller is to bedisplayed, along with other streams, at content presentation device 310.

If the input indicates that the current layout is to be changed (block608—YES), content merging device 102 may change the current layout(block 612). As indicated above, depending on the state of contentmerging device 102, video layout logic 502 in content merging device 102may select a particular layout. For example, when the second videophonecall is accepted, video layout logic 502 may select a layout in whichtwo windows display video streams from the callers side-by-side.Furthermore, video layout logic 502 may set the selected layout as thecurrent layout. In response, PIP module 408 may merge the receivedstreams in accordance with the current layout.

If the input indicates that the current layout is not to be changed(block 608—NO), content merging device 102 may perform a “cleanup”(block 610). In some instances, the cleanup may include erasing orremoving a text message that is displayed via content presentationdevice 310. The clean up may return content merging device 102 to astate prior to block 604.

In the above, process 600 illustrates changing a layout that isassociated with merging and displaying received video streams. Asdescribed, content merging device 102 may merge or combine the videostreams in accordance with a selected layout. FIGS. 7A and 7B illustratedifferent exemplary layouts for merging received video streams. Inparticular, FIGS. 7A and 7B illustrate exemplary layouts for videostreams during the video telepresence session described above withreference to FIGS. 2A through 2C.

FIG. 7A shows layouts of the output streams at digital televisions 110at Jacque and Sunny's homes 200 and 220 in accordance with processingdescribed with respect to FIG. 6. In Jacque's home 200, content mergingdevice 102 may select a layout that shows, in display area 702, TheThree Musketeers 222 and the video of Sunny. Sunny's content mergingdevice 102 may have merged The Three Musketeers 222 and the video ofSunny into a single stream and forward the merged stream to Jacque'scontent merging device 102 over network 112.

The layout may also include an area 704, which shows a video stream ofJacque from video camera 206. The streams from Sunny's content mergingdevice 102 and the camera are merged at Jacque's content merging device102, as shown in the left side of FIG. 7A.

In Sunny's home 220, content merging device 102 may select a layout thatshows the video stream of Jacque in area 228, Tigers in area 706, and avideo stream of Sunny from camera 226 in area 230. In this example, thevideo streams of Tigers 202 (which Jacque may or may not be watching atthat instant) and Jacque are sent as separate video streams from contentmerging device 102 at Jacque's home 200. The video streams of Tigers202, Sunny, and Jacque are merged at Sunny's content merging device 102,as shown on the right side of FIG. 7A.

FIG. 7B shows other layouts for output streams content merging devicesat Jacque and Sunny's homes 200 and 220. In Jacque's home 200, contentmerging device 102 may select a layout that shows only Tigers 202 fromset-top box 106. In Sunny's home 220, content merging device 102 maymerge the video stream of The Three Musketeers 222 from the DVR inSunny's home and the video stream from content merging device 102associated with Jacque's camera 206 at Jacque's home 200, as shown inarea 708.

In the above, content merging device 102 may merge one or more inputcontent streams into one or more output streams. Because of contentmerging device 102's interconnectivity to different media devices, usingcontent merging device 102 in network 300 may reduce the number ofcables that are connected to content presentation device 310. Inaddition, because content merging device 102 is separate from contentdelivery device 304, content merging device 102 may be manufactured andsold independently from the vendors of content delivery devices 304.Further, content merging device 102 may allow video telepresence, videoteleconference, and/or videophone service to be provided independentlyfrom content providers.

The foregoing description of implementations provides illustration, butis not intended to be exhaustive or to limit the implementations to theprecise form disclosed. Modifications and variations are possible inlight of the above teachings or may be acquired from practice of theteachings.

In addition, while series of blocks have been described with regard toan exemplary process illustrated in FIG. 6, the order of the blocks maybe modified in other implementations. In addition, non-dependent blocksmay represent acts that can be performed in parallel to other blocks.

It will be apparent that aspects described herein may be implemented inmany different forms of software, firmware, and hardware in theimplementations illustrated in the figures. The actual software code orspecialized control hardware used to implement aspects does not limitthe invention. Thus, the operation and behavior of the aspects weredescribed without reference to the specific software code—it beingunderstood that software and control hardware can be designed toimplement the aspects based on the description herein.

Further, certain portions of the implementations have been described as“logic” that performs one or more functions. This logic may includehardware, such as a processor, a microprocessor, an application specificintegrated circuit, or a field programmable gate array, software, or acombination of hardware and software.

No element, act, or instruction used in the present application shouldbe construed as critical or essential to the implementations describedherein unless explicitly described as such. Also, as used herein, thearticle “a” is intended to include one or more items. Further, thephrase “based on” is intended to mean “based, at least in part, on”unless explicitly stated otherwise.

What is claimed is:
 1. A device comprising: input components to receivemultiple input video streams, from different first external devices, theinput video streams including: video content of a user captured at afirst communication device for a multi-party video telecommunicationsession, established by a service provider, with a second communicationdevice, and multimedia content received from a content providerindependently of the video telecommunication session established by theservice provider; a first processor to: detect an event at the secondcommunication device during the video telecommunication session,independently of receiving the multimedia content front the contentprovider, determine a display layout in accordance with the event, andset the display layout as a current layout during the videotelecommunication session; a second processor to: combine, during thevideo telecommunication session, the video content and at least oneother of the multiple input video streams into a first output stream inaccordance with the current layout; and combine during the videotelecommunication session, the multimedia content and at least one otherof the multiple input video streams into a first output stream, whereinthe first output stream and the second output stream differ; outputcomponents to transmit the first output stream to second externaldevices associated with the second communication device for display inthe current layout during the video telecommunication session; and anetwork interface to transmit the second output stream, via a network,to the first communication device during the video telecommunicationsession.
 2. The device of claim 1, wherein the device comprises a videoconferencing device, a video telephone, or a video telecommunicationdevice.
 3. The device of claim 1, wherein the input components include:a video receiver for decoding high bandwidth digital content protection(HDCP) encoded stream.
 4. The device of claim 1, wherein the networkinterface includes at least one of: a WiFi interface or an Ethernetinterface.
 5. The device of claim 1, wherein the second processor isfurther configured to: generate a picture-in-picture video of combinedmultiple high definition video streams corresponding to the secondoutput stream.
 6. The device of claim 1, wherein the first output streamincludes a high definition multimedia interface (HDMI) stream.
 7. Thedevice of claim 1, further comprising a codec engine, wherein the firstprocessor is further configured to decode at least one of the. inputvideo streams via the codec engine and provide the decoded at least oneof the input video streams to the second processor to be merged withother input streams.
 8. The device of claim 7, wherein the firstprocessor is further configured to: send one or more of the input videostreams to the codec engine for encoding, and send the encoded one ormore of the input video streams to the output components.
 9. The deviceof claim 8, wherein the output components include a high speedcommunication interface via which the encoded one or more of the inputvideo streams are sent to one of the second communication device over anetwork.
 10. The device of claim 1, wherein the first processor isfurther configured to provide a graphical user interface (GUI) toreceive user input that selects the display layout.
 11. The device ofclaim 1, wherein the first external devices include at least one of: aset-top box, a computer, a digital video recorder (DVR) device, ahandset, or a cellular phone.
 12. The device of claim 1, wherein thesecond external devices include at least one of: a digital television ora computer.
 13. The device of claim 1, further comprising a double ratesynchronous dynamic random access memory coupled to the first processor.14. The device of claim 1, wherein the event includes at least one of: atermination of a videophone call or an incoming videophone call.
 15. Thedevice of claim 1, wherein the input video stream including themultimedia content includes a video stream received from a set-top box.16. A method comprising: detecting, by a first content merging device,an event at a first communication device via a multi-party videotelecommunication session, established by a service provider, with asecond communication device; determining, in response to the detecting,whether the event corresponds to a particular type of event; prompting,when the event corresponds to the particular type of event and duringthe video telecommunication session, a user of the first communicationdevice for user input; determining, by the first content merging deviceand during the video telecommunication session, a display layout inaccordance with user input received in response to the prompting;receiving multiple input video streams, by the first content mergingdevice and during the video telecommunication session, the input videostreams including: multimedia content provided by a content providerindependently of the video telecommunication session established by theservice provider, and video content of a user captured at the secondcommunication device for the video telecommunication session with thefirst communication device; merging, during the video telecommunicationsession, the video content and one or more other of the multiple inputvideo streams into a high definition output stream in accordance withthe display layout; merging, during the video telecommunication session,the multimedia content and one or more other of the multiple input videostreams into a second output stream, wherein the high definition outputstream and the second output stream differ; sending, during the videotelecommunication session, the high definition output stream to acontent presentation device associated with the first communicationdevice; and transmitting, via a network interface of the first contentmerging device and during the video telecommunication session, thesecond output stream to a second content merging device associated withthe second communication device.
 17. The method of claim 16, whereindetecting includes detecting an incoming videophone call and the methodfurther comprises: sending one of the input video streams to a caller ofthe videophone call.
 18. The method of claim 17, wherein sending the oneof the input video streams to the caller includes encoding the one ofthe input video streams.
 19. The method of claim 16, wherein the userinput includes: detecting a selection, by the user, that identifies thedisplay layout.
 20. A system comprising: media devices to send videostreams that include a high bandwidth digital content protection (HDCP)encoded stream, wherein at least one of the video streams includesmultimedia content from a content provider; a first content mergingdevice to: receive the video streams from the media devices, wherein atleast two of the media devices comprise communication devices thatcapture video of users via a video telecommunication session,established by a service provider, independently of the multimediacontent from the content provider; detect an event related to videocontent received from at least one of the communication devices duringthe video telecommunication session, determine, during the videotelecommunication session, a different display layout, corresponding todifferent ones of the communication devices, in accordance with theevent, combine, during the video telecommunication session, the videocontent and one or more of the received video streams into a highdefinition media interface (HDMI) output stream corresponding to thedifferent ones of the communication devices, in accordance with thedifferent display layouts, combine, during the video telecommunicationsession, the multimedia content and one or more other of the receivedvideo streams into a second output stream, wherein the HDMI outputstream and the second output stream differ, transmit, via a networkinterface of the first content merging device and during the videotelecommunication session, the second output stream to a second contentmerging device associated with the at least one of the communicationdevices, and transmit, during the video telecommunication session, thecorresponding HDMI output stream to a content presentation deviceassociated with the different ones of the communication devices; andwherein the content presentation devices display the corresponding HDMIoutput streams in the different display layouts during the videotelecommunication session.
 21. The system of claim 20, wherein thecontent presentation device includes: a computer or a digitaltelevision.